Repository navigation
feat(realtime): add opt-in Worker Realtime WebSocket transport - #62
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ❌ Deployment failed View logs |
multillm-proxy | c3712d1 | Oct 09 2026, 05:11 AM |
f0ceaf3
into
integrate/gateway-waves-1-5
Summary
Adds an opt-in Realtime WebSocket transport served entirely in the Worker. It bridges authenticated OpenAI-compatible sockets to an approved upstream and never wakes the Container. With
REALTIME_ENABLEDunset, empty, false or malformed (the default is off),handleRealtimeRequestreturnsnullbefore any other work, and every route, including/v1/realtime, keeps its existing handling byte for byte.REALTIME_PROVIDERS_JSONmaps an exactprovider:modelto a completewss://URL on a public DNS host, port 443. IP literals, local or internal names, userinfo, fragments and credential-like query fields are rejected.OpenAI-Beta: realtime=v1version header. Upstream headers are not returned.GET /v1/realtime?model=provider:modelwithUpgrade: websocketand a gateway key inAuthorization: BearerorX-MultiLLM-API-Key.chatandaudioscopes are required.no-store.POST /v1/realtime/client_secretsreturns a gateway-scoped ticket, never a provider secret.JWT_SECRET, lasts 60 seconds and is bound to one model and the issuing credential.REALTIME_PRICING_JSONgives per-model prices per million audio/text tokens and aprovider_session_limit_usd. That limit must be enforced by the provider account or session, and must not exceedREALTIME_SESSION_CAP_USD.REALTIME_DAILY_BUDGET_USDis required.Details are in
docs/realtime.md.Deviation from the brief
The brief asked for a monetary cap the gateway enforces. Usage events arrive after the upstream has already spent the money, so the gateway can't guarantee a cap on its own. Admission therefore requires a provider-enforced session ceiling no greater than the gateway cap, and refuses the session when there is none.
Limits to know
Authorizationheader. Browsers can't set headers on a WebSocket, and credentials in URLs or subprotocols are refused. So a plain browser client can't connect yet; server-side clients and native apps can.WebSocketPairand upstream sockets only. No live provider socket, billing or protocol compatibility has been tested.Wiring
cloudflare-worker.mjs:fetchgets one call,handleRealtimeRequest, right after the health check. A 101 response is returned as is; other responses get the existing CORS headers.worker/gateway-extensions.mjsre-exports the handler, andworker/api-paths.mjsaddsisRealtimeRequestPath.wrangler.jsoncis unchanged: no new bindings, Durable Object classes or vars.Shared registrations
tests/test_realtime_worker.mjsintest:worker.worker/realtime.mjs,worker/realtime-session.mjsandworker/realtime-metering.mjsin the edge module inventory.REALTIME_ENABLEDandREALTIME_PROVIDERS_JSONin the Container env keys.realtime_ticketsandrealtime_sessionsinREQUIRED_D1_TABLES.No Flask routes or browser-readable headers.
Tests
tests/test_realtime_worker.mjscovers:flag off, empty and malformed leaving every route unchanged, and flag on never reaching the Container;
the URL allowlist and that caller headers never reach upstream;
the 426, 401 and 403 responses, scopes, model grants and key controls;
ticket signing, expiry, single use, model binding, credential rotation and the no-reissue rule;
the 2-session limit, admission lease hold and release, TTL, idle and frame limits;
close-code coupling, upstream error forwarding and provider-key echo withholding;
price, ceiling and budget refusals before dialling;
measured settlement and unknown holds for partial or interrupted usage.
Full Python suite: 4,456 passed, 2 skipped. ruff, mypy and bandit are clean.
npm run test:worker: 1,656 passed.npm run test:integration: 7 passed.cf:migrations:checkreports 29 contiguous, additive migrations.test:deploy, free routes, client headers and thewrangler deploy --dry-runbundle all pass.tests/test_frontend_build.mjsfails locally because@tailwindcss/cliis not installed. It fails the same way on main.Apply the migration before enabling:
npx wrangler d1 migrations apply multillm-intelligence --remote.CI notes
python/nodejobs: the dependency-audit steps fail on every branch, including main. fix(deps): clear the CI dependency audit advisories #42 fixes that.AdmissionCoordinatorDurable Object migration, whichwrangler versions uploadcannot apply. This is expected untilwrangler deployruns on main.This is part of Wave 6. It is based on
integrate/gateway-waves-1-5, the Wave 5 integration branch (#60), and it is the base of the Wave 6 migration stack (0029). The README line comes with the Wave 6 integration PR.